Method of capturing linear features along a reference-line across a surface for use in a map database

ABSTRACT

A method of producing linear features along a reference-line across a surface for use in a map database is disclosed. In at least one embodiment, the method includes generating, from reference-line data representative of coordinates of the reference-line in a geographic coordinate reference system and source images of the surface adjacent to the reference-line and associated position and orientation data in the geographic coordinate reference system, a reference-line referenced data set, wherein the reference-line referenced data set includes a plurality of sets of image data and associated data defining a reference-line&#39; across a surface in the geographic coordinate reference system, the sets of image data including pixels wherein a set of image data corresponds to an orthorectified view representation of a line section of the surface in the geographic coordinate reference system, each set of image data includes a reference pixel being associated with a position on the reference-line, wherein each pixel represents a surface having a position at a distance from the position of the reference pixel along the line section, and wherein the line section perpendicularly crosses the reference-line at the position associated with the reference pixel; and, post processing the reference-line referenced data set to produce linear features along the reference-line and associated locations in the geographic coordinate reference system for use in a map database.

FIELD OF THE INVENTION

The present invention relates to a method of producing linear featuresalong a reference-line across a surface for use in a map database. Theinvention further relates to a method of generating a reference-linereferenced data set and a method of post processing a reference-linereferenced data set to produce linear features along a reference-lineacross a surface and associated locations in a coordinate system for usein a map database. Moreover, the invention relates to a reference-linereferenced data set, a computer implemented system for producing orverifying linear features, a computer program product and a processorreadable medium carrying said computer program product or reference-linereferenced data set.

PRIOR ART

There is a need to collect a large number of linear road informatione.g. lane dividers, road centrelines, road width etc. for digital mapdatabases used in navigation systems and the like. The geo-position ofthe road information could be stored as absolute or relative positioninformation. For example, the centreline could be stored with absolutegeo-position information and the road width could be stored withrelative position information, which is relative with respect to theabsolute geo-position of the centreline. The road information could beobtained by interpreting high resolution aerial orthorectified images.An orthorectified image is a “scale corrected” image, depicting groundfeatures as seen from above in their exact ground positions, in whichdistortion caused by camera and flight characteristics and reliefdisplacement has been removed using photogrammetric techniques. Anorthorectified image is a kind of aerial photograph that has beengeometrically corrected (“orthorectified”) such that the scale of thephotograph is uniform, meaning that the photograph can be consideredequivalent to a map. An orthorectified image can be used to measure truedistances, because it is an accurate representation of the earth'ssurface, having been adjusted for topographic relief, lens distortion,and camera tilt. Orthorectified views differ from perspective view as anorthorectified view project at a right angle to a reference plane,whereas perspective views project from the surface onto the referenceplane from a single fixed position. An orthorectified image can beobtained by any suitable map projection. The map projection can be aprojection by surface, such as cylindrical, pseudocylindrical, hybrid,conical, pseudoconical or azimuthal. The projection can also be aprojection by preservation of a metric property. The map projectionshave in common that they are orthogonal projections, which means thatevery pixel represents a point on the surface of the reference plane(ellipsoid that approximates the shape of the earth) seen along a lineperpendicular to that surface. Thus, every pixel of an orthorectifiedimage of the earth surface substantially corresponds to a view of theearth surface seen along a line perpendicular to the ellipsoid thatapproximates the shape of the earth. An orthorectified image comprisesmetadata enabling an algorithm to reference any pixel of theorthorectified image to a point in the geographic coordinate referencesystem. As the exact position on the ellipsoid that approximates theshape of the earth of each pixel is known, the position and size ofground features, e.g. horizontal road information, can be retrieved froman orthorectified image. Such high resolution orthorectified imagesshould have a pixel size below 25 cm.

Nowadays, “vertical” road information, e.g. speed limits, directionssignposts etc. for digital map databases used in navigation systems andthe like, can be obtained by analysing and interpreting horizontalpicture images and other data collected by a earth-bound mobilecollection device. The term “vertical” indicates that an informationplane of the road information is generally parallel to the gravityvector. Mobile mapping vehicles which are terrestrial based vehicles,such as a car or van, are used to collect mobile data for enhancement ofdigital map databases. Examples of enhancements are the location oftraffic signs, route signs, traffic lights, street signs showing thename of the street etc.

The mobile mapping vehicles have a number of cameras, some of themstereographic and all of them are accurately geo-positioned as a resultof the van having precision GPS and other position determinationequipment onboard. While driving the road network, image sequences arebeing captured. These can be either video or still picture images.

The mobile mapping vehicles record more then one image in an imagesequence of an object, e.g. a building or road surface, and for eachimage of an image sequence the geo-position in a geographic coordinatereference system is accurately determined together with the position andorientation data of the image sequence with respect to saidgeo-position. Image sequences with corresponding geo-positioninformation will be referred to as geo-coded image sequences. As theimages sequences obtained by a camera represent a visual perspectiveview of the ‘horizontal” road information, image processing algorithmsmight provide a solution to extract the road information from the imagesequences. The geo-positions of the cameras and laser scanners areaccurately known by means of an onboard positioning system (e.g. a GPSreceiver) and other additional position and orientation determinationequipment (e.g. Inertial Navigation System—INS). The geo-coded imagesequences are used to generate orthorectified images. Geo-coded meansthat a position, computed by the GPS and possibly INS, and possiblyheading associated with the image is attached to the metadata of theimage. An geo-coded orthorectified image comprises metadata to definethe associated projected coordinate reference system to determine foreach pixel the corresponding position in the geographic coordinatereference system. These orthorectified images are comparable to aerialorthorectified images but with improved resolution and absolutegeo-position. A method of generating orthorectified images andassociated metadata defining the position and orientation data of theorthorectified image is disclosed in unpublished patent applicationPCT/NL2006/050252. This method enables us to generate very accurategeo-coded orthorectified images from Mobile Mapping System data only.The geo-coded images have a pixel resolution of 8 cm (=relative positionaccuracy within the image) and the metadata defining the position andorientation of the image on the earth surface has an absolutegeo-position accuracy of 1 meter.

In both the aerial images and image sequences, the horizontal roadinformation is present. By means of complex image processing algorithms,the lane information can be detected and the corresponding positioninformation can be determined. Furthermore, a human can mark the desiredhorizontal road information in the orthorectified images on a screen.Software then calculates from said marks the geo-positions in thegeographic coordinate reference system of the horizontal roadinformation. Marking linear road information of curved roads, e.g. theroad sides are in a bend, is more time consuming then marking the roadside along a straight road. To mark a straight line only two points haveto be indicated whereas to mark a curve more points have to beindicated. The accuracy of the curve depends on the number of points onthe curve selected. Therefore, the more curvy a road the more time iseffort is needed to retrieve the horizontal road information from theimages. Similarly, the more curvy a road the more complex algorithms areneeded to retrieve the horizontal road information from orthorectifiedimages. First, the curved road surface has to be identified in thegeo-coded orthorectified image. Secondly, the road markings have to befound on the road surface in the image. Finally, the linear roadmarkings along the road have to be digitized. As the position andorientation of the road are not know in a geo-coded orthorectifiedimage, symmetrical filters such as morphology filters have to be used todetect the linear road markings automatically. These filters are morecomplex then filters for detecting vertical or horizontal lines/edgesonly.

Furthermore, there are millions of kilometers or roads in the world thatin the near future will need to be ADAS (Advanced Driver AssistanceSystems) compliant. ADAS applications such as Adaptive Cruise Control,Transmission Assistance, Lane/Road Departure Detection and Warning,Braking and Stability Control Assistance, needs highly accurate roadinformation about the road to control the vehicle and or warn the driverof safety issues. The road information needs to be captured and storedin a map database. The digital map will provide knowledge of the roadahead of the moving vehicle. It will provide prediction capabilities ofup-coming curve beyond the driver's line of sight, such as curve shape,curve direction and radius, and road characteristics such as road type(high way, highway ramp, local road, etc.), number of lanes, width ofroad, type of road, lane markings, etc. The applications will use theinformation to inform the driver of coming situations and to assist himto drive safe. ADAS quality precision requires zooming to have imageswherein 1 pixel on a screen represents an area of 10×10 cm. This meansthat a screen having 1280 by 1024 pixels, can show an area of 128 by102.4 meters. To digitize a straight road of 1 km requires a human toscroll every 100 m and to digitize one point. Scrolling means that theimage has to be refreshed from the image database. Both refreshing theimage and digitizing one point are time consuming.

Therefore, methods are needed which can accurately, efficiently andrapidly extract horizontal road features and corresponding positioninformation from aerial orthorectified images, satellite images, aerialimage sequences or image sequences captured by a camera mounted on amoving vehicle. There is a need for algorithms to speed up road geometryupdate and acquisition of new road geometry.

DEFINITIONS

-   Coordinate: one of a sequence of n numbers designating the position    of a point in n-dimensional space;-   Coordinate conversion: change of coordinates, based on a one-to-one    relationship, from one coordinate reference system to another based    on the same datum;-   Coordinate reference system: coordinate system which is related to    the real world by a datum;-   Coordinate system: set of mathematical rules for specifying how    coordinates are to be assigned to points;-   Datum: parameter or set of parameters that define the position of    the origin, the scale, and the orientation of a coordinate reference    system;-   Ellipsoidal coordinate system (geodetic coordinate system):    coordinate system in which position is specified by geodetic    latitude, geodetic longitude and (in three-dimensional case)    ellipsoidal height, associated with one or more geographic    coordinate reference systems;-   Geographic coordinate reference system: coordinate reference system    using an ellipsoidal coordinate system and based on an ellipsoid    that approximates the shape of the earth;-   Map Projection: coordinate conversion from an ellipsoidal coordinate    system to a plane;-   Orthorectified view: view of a point from a chosen reference surface    along a line perpendicular to that surface in said point;-   Projected coordinate reference system: coordinate reference system    derived from a two-dimensional geographic coordinate reference    system by applying a map projection and using a Cartesian coordinate    system.

SUMMARY OF THE INVENTION

The present invention seeks to provide an improved method of capturingor verifying linear features along a reference-line across a surface foruse in a map database.

According to the present invention, the method comprises:

-   -   generating from reference-line data representative of        coordinates of said reference-line in a geographic coordinate        reference system and source images of the surface adjacent to        said reference-line and associated position and orientation data        in said geographic coordinate reference system, a reference-line        referenced data set, wherein the reference-line referenced data        set comprises a plurality of sets of image data and associated        data defining a reference-line across a surface in the        geographic coordinate reference system, the sets of image data        having pixels wherein a set of image data corresponds to an        orthorectified view representation of a line section of the        surface in the geographic coordinate reference system, each set        of image data comprises a reference pixel being associated with        a position on the reference-line, wherein each pixel represents        a surface having a position at a predefined distance from the        position of the reference pixel along the line section, and        wherein the line section perpendicularly crosses the        reference-line at the position associated with the reference        pixel; and,    -   post processing the reference-line referenced data set to        produce linear features along the reference-line and associated        locations in the geographic coordinate reference system for use        in a map database.

The invention is based on the recognition that linear road information,such as lane dividers, road sides, road centerline etc., is roadinformation which is parallel to the direction of the road. Thedirection of a road can be obtained from a map database. Furthermore,the driving direction of a vehicle on a road is a good approximation ofthe direction of a road. The position and orientation of the vehicle isdetermined by means of a GPS receiver and an inertial measuring device,such as one or more gyroscopes and/or accelerometers. In this way atrack-line of the vehicle can be made in a geographic coordinatereference system. The track-line is a good approximation of thedirection of the road. On an orthorectified image, a road and thus thetrack-line of a vehicle or linear road information will be curved. Theidea is to transform the area along a reference-line, for example thetrack-line of a vehicle, in the orthorectified image into areference-line referenced image wherein the reference-line and areasparallel to the reference-line are a straight line and rectangles alongthe straight line. In this way, a curved road with constant width willbe transformed into an image of an almost straight road with almostparallel road sides, a reference-line according to a centerline or lanemarkings will be almost straight lines in a reference-line referencedimage. The degree of straightness of the linear features in thereference-line referenced image depends on the accuracy of the positionof the reference-line with respect to the linear features. If thevehicle drives always in the middle of one lane, the linear featureswill be substantially straight. However, if the vehicle changes lanes,he will cross a lane divider, this will be seen as a horizontal changeof the lane divider in subsequent rows of pixels in the reference-linereferenced image. As stated above the linear features in such an imageare much easier and faster to digitize or verify by a human then wouldbe the case when using the original orthorectified image. As theperformed transformation is reversible, the exact geo-position of alinear feature can be derived from the positions of a feature determinedin the reference-line referenced (straight line) image. Theorthorectified geo-referenced space to reference-line referenced imagespace (RRI Space) transformation allows a human to analyze at least 20times more information at one time. Another advantage of areference-line referenced image is that the human analyzing the roadsurface only needs to scroll up and down in the image to move along theroad surface, whereas he has to scroll in 2-dimensions in the image ifhe would like to move along the road on an orthorectified image. In thisway, the human can move fast along the road surface to located arequired position.

Furthermore, by transforming a curved road into a straight road, lesscomplicated image processing algorithms are needed to detect linear roadinformation in the reference-line referenced images. If thereference-line is projected on a column of pixels in the reference-linereferenced image, the linear road information will appear as verticalinformation in the reference-line referenced images. The algorithms onlyneeds to find vertical information in the image.

By means of the invention, the linear road information of a particulararea, e.g. roads within a region, can be captured or edited much faster.This reduces the cost to manufacture the information, but also reducesthe time between capturing the images and providing the road informationto a customer. The invention further enables digital map providers toprovide more frequently updates of a digital map.

According to the invention a method of generating a reference-linereferenced data set comprises:

-   -   acquiring reference-line data representative of the coordinates        of said reference-line in a geographic coordinate reference        system;    -   acquiring source images of a surface adjacent to said        reference-line and associated position and orientation data in        said geographic coordinate reference system;    -   generating a sequence of sample positions on the reference-line;    -   determining for each sample position, the location of a line        section perpendicular to the direction of the reference-line at        said sample position;    -   generating a line of pixels for each line section, wherein each        pixel has an associated pixel position in the geographic        coordinate reference system on the surface and the pixel value        has been derived from at least one pixel of the source images        representative of said associated pixel position;    -   storing the lines of pixels and associated sample positions in        the reference-line referenced data set.

The source images could be taken from one of the group: images capturedby a terrestrial camera mounted on a moving vehicle, aerial images,satellite images, orthorectified images. The reference-line could betaken from one of the group: track line of the vehicle, road centerlinefrom existing database, other existing road geometry taken from anexisting database.

In an embodiment of the invention, the sample positions are equidistantalong the reference-line. This feature allows a software program toprovide without transformation a reference-line referenced image whichis easy to interpret by a human.

In another embodiment of the invention, the distance along thereference-line between two subsequent sample positions in the geographiccoordinate reference system depends on the local curvature orstraightness of the reference-line. This feature enables the method toprovide a reference-line referenced data set with an optimal amount ofdata. Fewer points are needed to accurately derive the linear roadinformation on straight sections of a road then on bended sections.

According to the invention a method of post processing a reference-linereferenced data set to produce linear features along a reference-lineacross a surface and associated locations in a geographic coordinatereference system for use in a map database comprises:

-   -   retrieving the reference-line referenced data set, wherein the        reference-line referenced data set comprises a plurality of sets        of image data and associated data defining a reference-line        across a surface in a geographic coordinate reference system,        the sets of image data having pixels wherein a set of image data        corresponds to an orthorectified view representation of a line        section of the surface in the geographic coordinate reference        system, each set of image data comprises a reference pixel being        associated with a position on the reference-line, wherein each        pixel represents a surface having a position at a predefined        distance from the position of the reference pixel along the line        section, and wherein the line section perpendicularly crosses        the reference-line at the position associated with the reference        pixel;    -   transforming the reference-line referenced data set into a        reference-line referenced image wherein

one or more source images to obtain a transformed image in dependence ofthe road position information, wherein each column of pixels of thetransformed image corresponds to a surface parallel to the direction ofsaid road;

-   -   selecting a linear feature in the reference-line referenced        image;    -   determining coordinates in the geographic coordinate reference        system of the linear feature from the position of the linear        feature in the reference-line referenced image and associated        data;    -   storing the coordinates of the linear feature in a database.

The linear feature is one of the group: road centerline, road width,road curb, road painting, lane divider, number of lanes, traffic island,junctions and any other visual distinguishing feature of the surfacealong the reference-line.

In an embodiment of the invention selecting comprises:

-   -   outputting the reference-line referenced image on a screen;    -   manually positioning a pointing device on the linear feature on        the screen to obtain marked positions; and

determining calculates the coordinates in the geographic coordinatesystem of the linear feature from the marked positions.

These features allow us to use humans to analyze the reference-linereferenced images and to produce the linear road information.

In another embodiment selecting comprises:

-   -   performing a line detection algorithm on the reference-line        referenced image to select the linear feature;    -   determining positions of linear feature in reference-line        referenced image; and,

determining calculates the coordinates in the geographic coordinatesystem of the linear feature from the pixel positions of the linearfeature in the reference-line referenced image.

These features allow us to produce automatically the linear roadinformation from the reference-line referenced data sets.

In another aspect, the invention provides a reference-line referenceddata set comprising a plurality of sets of image data and associateddata defining a reference-line across a surface in a geographiccoordinate reference system, the sets of image data having pixelswherein a set of image data corresponds to an orthorectified viewrepresentation of a line section of the surface in the geographiccoordinate reference system, each set of image data comprises areference pixel being associated with a position on the reference-line,wherein each pixel represents a surface having a position at apredefined distance from the position of the reference pixel along theline section, and wherein the line section perpendicularly crosses thereference-line at the position associated with the reference pixel.

In an embodiment of the reference-line referenced data set the surfaceis the earth surface and a line section includes a section of a surfaceof a road. In an embodiment of the reference-line referenced data set,the plurality of sets of image data is a reference-line referencedimage, wherein each set of image data corresponds to a row of pixels ofthe reference-line referenced image. In another embodiment, thepositions of the pixels of a set of image data are proportionallydistributed along the line section when projected on the geographiccoordinate reference system. In yet another embodiment, thereference-line corresponds to an approximation of a road centerline.

And in a further embodiment, the geographic coordinate reference systemis the WGS84 coordinate system.

In another aspect, the invention provides a computer implemented systemfor verifying linear features along a reference-line and associatedlocations in a geographic coordinate reference system, the systemcomprising:

-   -   an input device;    -   a processor readable storage medium; and    -   a processor in communication with said input device and said        processor readable storage medium;    -   an output device to enable the connection with a display unit;        said processor readable storage medium storing code to program        said processor to perform a method comprising the actions of:    -   retrieving from a map database reference-line data        representative of coordinates of a reference-line across a        surface in a geographic coordinate reference system;    -   retrieving an orthorectified image of said surface and        associated position and orientation data in said geographic        coordinate reference system;    -   generating from the reference-line data and orthorectified        images a reference-line referenced image, wherein each row of        pixels of the referenced-line referenced image corresponds to a        section of said surface perpendicular to the direction of the        reference-line and each column of pixels of the reference-line        referenced image corresponds to a surface parallel to the        reference-line;    -   verifying the position of linear features in the reference-line        referenced image;    -   marking positions showing defects with respect to at least one        requirement taken from the group: position of linear feature,        straightness of linear feature, parallelism of linear features;    -   determining coordinates in the geographic coordinate reference        system corresponding to the marked positions showing defects;        and,    -   storing the coordinates of the marked defects in a database for        further processing.

Reference-line referenced images according to the invention, enableshumans to verify efficiently and easily the positions of linear featuresin existing map databases. The position of a linear feature, e.g. lanedivider, taken from an existing map database can be used asreference-line. The orthorectified images visualizes the real surface ofthe earth when projected on the ellipsoid that approximates the shape ofthe earth used by a geographic coordinate reference system. In thereference-line referenced image, the linear feature should be visual ona predefined position in the image. Furthermore, a linear feature shouldbe a straight line. If the position is incorrect or the linear featureis not a straight line, a human will recognize this as a defect andcould mark the defect. Subsequently, the geo-position of the defect isdetermined by the projected coordinate reference system associated withthe pixels of the reference-line referenced image. The geo-positions ofthe defects can be used to retrieve corresponding source images from amobile mapping vehicle or a corresponding orthorectified image, todigitize again the linear feature in the images with relative highresolution. In this way, data in existing map databases can be verifiedefficiently and corrected if necessary.

The present invention can be implemented using software, hardware, or acombination of software and hardware. When all or portions of thepresent invention are implemented in software, that software can resideon a processor readable storage medium. Examples of appropriateprocessor readable storage medium include a floppy disk, hard disk, CDROM, DVD, memory IC, etc. When the system includes hardware, thehardware may include an output device (e.g. a monitor, speaker orprinter), an input device (e.g. a keyboard, pointing device and/or amicrophone), and a processor in communication with the output device andprocessor readable storage medium in communication with the processor.The processor readable storage medium stores code capable of programmingthe processor to perform the actions to implement the present invention.The process of the present invention can also be implemented on a serverthat can be accessed over telephone lines or other network or internetconnection.

SHORT DESCRIPTION OF DRAWINGS

The present invention will be discussed in more detail below, using anumber of exemplary embodiments, with reference to the attached drawingsthat are intended to illustrate the invention but not to limit its scopewhich is defined by the annexed claims and its equivalent embodiment, inwhich

FIG. 1 shows a MMS system with a camera and a laser scanner;

FIG. 2 shows a diagram of location and orientation parameters;

FIG. 3 shows a block diagram of a computer arrangement with which theinvention can be performed;

FIG. 4 is a flow diagram of an exemplar implementation of the processfor producing linear road information according to the invention;

FIG. 5 illustrates the transformation from orthorectified image space toreference-line referenced image space;

FIG. 6 illustrates the generation of an image data set;

FIG. 7 illustrates the generation of a reference-line referenced imagefrom an orthorectified image;

FIGS. 8-10 illustrate examples of the transformation from orthorectifiedimage space to reference-line referenced image space with differentreference-lines;

FIG. 11 illustrates the effectiveness of the invention;

FIG. 12 illustrates the transformation from reference-line referencedspace to orthorectified space; and,

FIG. 13 illustrates the effect of decreasing the number image lines inreference-line referenced image space.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 shows a MMS system that takes the form of a car 1. The car 1 isprovided with one or more cameras 9(i), i=1, 2, 3, . . . I. The lookingangle or the one or more cameras 9(i) can be in any direction withrespect to the driving direction of the car 1 and can thus be a frontlooking camera, a side looking camera or rear looking camera, etc. Theviewing window(s) of the camera(s) 9(i) cover(s) the whole road surfacein front the vehicle. Preferably, the angle between the drivingdirection of the car 1 and the looking angle of a camera is within therange of 45 degree-135 degree on either side. The car 1 can be driven bya driver along roads of interest.

The car 1 is provided with a plurality of wheels 2. Moreover, the car 1is provided with a high accuracy position determination device. As shownin FIG. 1, the position determination device comprises the followingcomponents:

-   -   a GPS (global positioning system) unit connected to an antenna 8        and arranged to communicate with a plurality of satellites SLi        (i=1, 2, 3, . . . ) and to calculate a position signal from        signals received from the satellites SLi. The GPS unit is        connected to a microprocessor μP. Based on the signals received        from the GPS unit, the microprocessor μP may determine suitable        display signals to be displayed on a monitor 4 in the car 1,        informing the driver where the car is located and possibly in        what direction it is traveling. Instead of a GPS unit a        differential GPS unit could be used. Differential Global        Positioning System (DGPS) is an enhancement to Global        Positioning System (GPS) that uses a network of fixed ground        based reference stations to broadcast the difference between the        positions indicated by the satellite systems and the known fixed        positions. These stations broadcast the difference between the        measured satellite pseudoranges and actual (internally computed)        pseudoranges, and receiver stations may correct their        pseudoranges by the same amount.    -   a DMI (Distance Measurement Instrument). This instrument is an        odometer that measures a distance traveled by the car 1 by        sensing the number of rotations of one or more of the wheels 2.        The DMI is also connected to the microprocessor μP to allow the        microprocessor μP to take the distance as measured by the DMI        into account while calculating the display signal from the        output signal from the GPS unit.    -   an IMU (Inertial Measurement Unit). Such an IMU can be        implemented as 3 gyro units arranged to measure rotational        accelerations and translational accelerations along 3 orthogonal        directions. The IMU is also connected to the microprocessor μP        to allow the microprocessor μP to take the measurements by the        DMI into account while calculating the display signal from the        output signal from the GPS unit. The IMU could also comprise        dead reckoning sensors.

It will be noted that one skilled in the art can find many combinationsof Global Navigation Satellite systems and on-board inertial and deadreckoning systems to provide an accurate location and orientation of thevehicle and hence the equipment (which are mounted with know positionsand orientations with references to the vehicle).

The system as shown in FIG. 1 is a so-called “mobile mapping system”which collects geographic data, for instance by taking pictures with oneor more camera(s) 9(i) mounted on the car 1. The camera(s) are connectedto the microprocessor μP. The camera(s) 9(i) in front of the car couldbe a stereoscopic camera. The camera(s) could be arranged to generate animage sequence wherein the images have been captured with a predefinedframe rate. In an exemplary embodiment one or more of the camera(s) arestill picture cameras arranged to capture a picture every predefineddisplacement of the car 1 or every interval of time. The predefineddisplacement is chosen such that a location at a predefined distance infront of the vehicle is captured be at least two subsequent pictures ofa camera. For example a picture could be captured after each 4 meters oftravel, resulting in an overlap in each image of a the road surfaceplane 5 meters distance heading the vehicle.

It is a general desire to provide as accurate as possible location andorientation measurement from the 3 measurement units: GPS, IMU and DMI.These location and orientation data are measured while the camera(s)9(i) take pictures. The pictures are stored for later use in a suitablememory of the μP in association with corresponding location andorientation data of the car 1, collected at the same time these pictureswere taken. The pictures include visual information as to roadinformation, such as center of road, road surface edges and road width.

FIG. 2 shows which position signals can be obtained from the threemeasurement units GPS, DMI and IMU shown in FIG. 1. FIG. 2 shows thatthe microprocessor μP is arranged to calculate 6 different parameters,i.e., 3 distance parameters x, y, z relative to an origin in apredetermined coordinate system and 3 angle parameters ω_(x), ω_(y), andω_(z), respectively, which denote a rotation about the x-axis, y-axisand z-axis respectively. The z-direction coincides with the direction ofthe gravity vector. The global UTM or WGS84 coordinate system could beused as predetermined geographic coordinate reference system.

The microprocessor in the car 1 and memory 9 may be implemented as acomputer arrangement. An example of such a computer arrangement is shownin FIG. 3.

In FIG. 3, an overview is given of a computer arrangement 300 comprisinga processor 311 for carrying out arithmetic operations. In theembodiment shown in FIG. 1, the processor would be the microprocessorμg.

The processor 311 is connected to a plurality of memory components,including a hard disk 312, Read Only Memory (ROM) 313, ElectricalErasable Programmable Read Only Memory (EEPROM) 314, and Random AccessMemory (RAM) 315. Not all of these memory types need necessarily beprovided. Moreover, these memory components need not be locatedphysically close to the processor 311 but may be located remote from theprocessor 311.

The processor 311 is also connected to means for inputting instructions,data etc. by a user, like a keyboard 316, and a mouse 317. Other inputmeans, such as a touch screen, a track ball and/or a voice converter,known to persons skilled in the art may be provided too.

A reading unit 319 connected to the processor 311 is provided. Thereading unit 319 is arranged to read data from and possibly write dataon a removable data carrier or removable storage medium, like a floppydisk 320 or a CDROM 321. Other removable data carriers may be tapes,DVD, CD-R, DVD-R, memory sticks etc. as is known to persons skilled inthe art.

The processor 311 may be connected to a printer 323 for printing outputdata on paper, as well as to a display 318, for instance, a monitor orLCD (liquid Crystal Display) screen, or any other type of display knownto persons skilled in the art.

The processor 311 may be connected to a loudspeaker 329.

Furthermore, the processor 311 may be connected to a communicationnetwork 327, for instance, the Public Switched Telephone Network (PSTN),a Local Area Network (LAN), a Wide Area Network (WAN), the Internet etcby means of I/O means 325. The processor 311 may be arranged tocommunicate with other communication arrangements through the network327. The I/O means 325 are further suitable to connect the positiondetermining device (DMI, GPS, IMU), camera(s) 9(i) and laser scanner(s)3(j) to the computer arrangement 300.

The data carrier 320, 321 may comprise a computer program product in theform of data and instructions arranged to provide the processor with thecapacity to perform a method in accordance to the invention. However,such computer program product may, alternatively, be downloaded via thetelecommunication network 327.

The processor 311 may be implemented as a stand alone system, or as aplurality of parallel operating processors each arranged to carry outsubtasks of a larger computer program, or as one or more main processorswith several sub-processors. Parts of the functionality of the inventionmay even be carried out by remote processors communicating withprocessor 311 through the telecommunication network 327.

The components contained in the computer system of FIG. 3 are thosetypically found in general purpose computer systems, and are intended torepresent a broad category of such computer components that are wellknown in the art.

Thus, the computer system of FIG. 3 can be a personal computer,workstation, minicomputer, mainframe computer, etc. The computer canalso include different bus configurations, networked platforms,multi-processor platforms, etc. Various operating systems can be usedincluding UNIX, Solaris, Linux, Windows, Macintosh OS, and othersuitable operating systems.

For post-processing the images and scans as taken by the camera(s) 9(i)and position/orientation data; a similar arrangement as the one in FIG.3 will be used, be it that that one will not be located in the car 1 butmay conveniently be located in a building for off-line post-processing.The images as taken by camera(s) 9(i) and associatedposition/orientation data are stored in one or more memories 312-315.That can be done via storing them first on a DVD, memory stick or thelike, or transmitting them, possibly wirelessly, from the memory 9. Theassociated position and orientation data, which defines the track of thecar 1 could be stored as raw data including time stamps. Furthermore,each image has a time stamp. The time stamps enables us to determineaccurately the position and orientation of the camera(s) 9(i) at theinstant of capturing an image. In this way the time stamps define thespatial relation between views shown in the images. The associatedposition and orientation data could also be stored as data which islinked by the used database architecture to the respective images. Ageo-coded image sequences is the combination of image sequences andassociated position and orientation data. The geo-coded image sequencesare used to generated orthorectified images. These orthorectified imagesare comparable to aerial orthorectified images but with improvedresolution and absolute geo-position. A method of generatingorthorectified images and associated position and orientation data isdisclosed in unpublished patent application PCT/NL2006/050252. A typicalMobile Mapping System MMS produces with the method an orthorectifiedmosaic that has a 8 cm resolution with an absolute position accuracy of1 m.

The present invention can use as image data any orthorectified imagewith sufficient resolution and accuracy. Thus orthorectified aerialimages, orthorectified satellite images as well as orthorectified imagesobtained from images captured by a camera mounted on a moving vehicle.It should be noted that the raw image sequences captured by a mobilemapping system could also be used as input data. This will be explainedin more detail below.

FIG. 4 is a flow diagram of an exemplar implementation of the processfor producing linear road information according to the invention. Linearroad information could be and is not limited to the road width, positionof road centerline, number of lanes, position, type and dimensions oflane dividers, width of lanes, position and dimensions of trafficislands, position and dimensions of parking/emergency stops, positionand dimensions of junctions with exit lane along the road. Input datafor the process are reference-line data 402 and image data 404. Thereference-line data corresponds to the reference-line across the earthsurface. The reference-line is preferably specified in a standardgeographic coordinate reference system, for example the WGS84 geographiccoordinate reference system. The reference-line could be the track of amobile mapping vehicle during an Mobile Mapping Session. If so, theimage sequences captured and the reference-line will have position andorientation data generated by the same position determination device.This provides an exact match between the position of the reference-lineand information retrieved from the images. However, the reference-linedata could also be extracted from a digital map database. Thereference-line could correspond to the route determined by a routeplanning system from position a to position b. The image data 404 couldany orthorectified image with metadata defining the correspondingprojected coordinate reference system. Consequently, every pixel of theimage can be mapped to real world coordinates and any world coordinatecan be found on the image. For the process according to the invention itis important that the projected coordinate reference system associatedwith the pixels of the orthorectified image and the projected coordinatereference system associated with the pixels of the reference-linereferenced data set enable a program to project a pixel of thereference-line referenced data 402 on a pixel of the image data 404. Theimage data is used to determine linear road features along thereference-line. If the position information does not match the method isnot able to determine the relevant areas in the images encompassing thelinear road feature. A mobile mapping vehicle drives on a road.Therefore, the track line of the mobile mapping vehicle is a goodapproximation of the position of the road. If the image sequencescaptured by the mobile mapping vehicle are used to generate anorthorectified image if the road, the track line of the vehicle canexactly be determined in the orthorectified image. Thus a reference-lineaccording to the track line of a mobile mapping vehicle enables us todetermine where to find in the orthorectified image the parts whichcomprise the road information we are looking for.

As stated above, the reference-line data is used to determine the partsof the orthorectified image to be analyzed. In action 406, the parts ofthe orthorectified image to be analyzed are transformed fromorthorectified image space to reference-line referenced image space. Asa result of this transformation, the curved reference-line inorthorectified image space is transformed to a straight line inreference-line referenced image space. Furthermore, locations in theorthorectified image space at a perpendicular distance d_(ort) from thereference-line are projected at locations at a perpendicular distanced_(RRI) from the referenced-line in the reference-line referenced imagespace. Consequently, all locations at a perpendicular distance d_(ort)are projected on a straight line parallel to the reference-line in thereference-line referenced image space at distance d_(RRI). Thistransformation will be described in more detail in the descriptionbelow. Output of action 406 are reference-line referenced data sets 408.A reference-line referenced data set comprises a plurality of sets ofimage data and associated position data defining a reference-line acrossa surface in a coordinate system, e.g. geographic coordinate referencesystem such as WGS84 and UTM. Each set of image data consists of alinear array of pixels and metadata defining the associated projectedcoordinate reference system. A linear array of pixels can be a line ofpixels of a reference-line referenced image. The linear array of pixelscorresponds to a orthorectified view representation of a line section ofthe surface in the coordinate system. An orthorectified image is anorthorectified view representation of the earth surface. It should benoted that an orthorectified image comprises metadata defining the usedmap projection. The map projection defines the transformation of thetwo-dimensional curved surface model of the earth to a plane. Anorthorectified image is a suitable image to generate the linear array ofpixels as the geo-position (the position of the two-dimensional curvedsurface of the earth) of each pixel is clearly defined. Each set ofimage data comprises a reference pixel being associated with a positionon the reference-line. Furthermore, each pixel represents a surfacehaving a position at a predefined distance from the position at saidsurface of the reference pixel along the line section. The line sectionperpendicularly crosses the reference-line at the position associatedwith the reference pixel. When an orthorectified image is the sourceimage, the values of the linear array of pixels correspond to the valuesof a line section in the orthorectified image which is perpendicular tothe reference-line at a defined location in the orthorectified image.Positions on the reference-line are defined. For each position on thereference-line, corresponding image data is derived from theorthorectified image. The combination of the image data in the lineararrays results in a reference-line referenced image. Block 410 indicatesthe combination of the image data and metadata enabling the calculationof the geo-position of each pixel to generate a reference-linereferenced image with metadata enabling the calculation of thegeo-position of each pixel accurately.

As state above the reference-line referenced data set comprises data tocompose a reference-line referenced image. In a reference-linereferenced image each column of pixels corresponds to a curve in theorthorectified image which is parallel to the reference-line. Thepresent invention is used to produce linear road information. This isinformation which is substantially parallel to the reference line, e.g.the direction of the road. By the transformation from orthorectifiedimage to reference-line referenced image, a curved road with constantwidth is transformed in a straight road with constant width. FIG. 5illustrates the transformation from orthorectified image space toreference-line referenced image space. The left hand image shows anorthorectified images of a bend of a road 502. Furthermore, the trackline of a vehicle 506 is indicated. Furthermore, a vector 510 indicatesthe direction of the reference-line in a point 509 and a vector 512defines the direction of a line section perpendicular to the directionof the reference-line 506. The right hand image shows the result of theorthorectified image space to reference-line referenced image spacetransformation of the left hand image. It can be seen that thereference-line corresponds to a vertical line 508 in the reference-linereferenced image and the curved centerline and road sides in theorthorectified image are transformed to vertical lines parallel to thevertical line 508.

Block 412 indicates the method to select linear features from thereference-line referenced image and the xy-position in the image. Themethod can be done by a human who digitizes the image. A human will seean almost straight road on the screen. By means of pointing means, suchas a pointer, the human can mark the position of the linear features inthe screen. The marked position in the linear reference image on thescreen is stored for further processing. The number of marks to select alinear feature depends on the straightness of the linear feature in thelinear reference image. The straightness of the linear feature dependson the parallelism of the reference-line and the feature in theorthorectified image. For example, if the moving vehicle is movinglanes, this movement is visible as horizontal movement of the feature inthe reference-line referenced image. To select accurately the linearfeature in the reference-line referenced image, the marks added by thehuman has to follow this movement. If the reference-line is parallel tothe linear feature in the orthorectified image, only the begin and endpoint of the linear feature have to be marked in the reference-linereferenced image.

Block 412 can also be implemented by an image processing algorithm.There are numerous algorithms know to the humans skilled in the art todetect linear features in images. As the used algorithm is not anessential feature of the invention, further details of such an algorithmare not described. As in a reference-line referenced image the linearfeatures to be detected appear as lines having an angle with respect toa column of pixels smaller than 45 degrees, which is in an embodimentsmaller than 5 degrees, simple line detection algorithms can be used todetect the linear features. If the reference-line corresponds to thetrack-line of a mobile mapping vehicle, the linear features will bemainly shown as vertical lines in the reference-line referenced images.

Block 412 can also be a method for verifying linear features along areference-line and associated locations in a coordinate system. In thatcase, the reference-line could correspond to the positions of any linearfeature retrieved from an existing digital map database. The positionsare used to generate the reference-line referenced image. Theorthorectified images or raw source images from a mobile mapping sessionvisualize the real surface of the earth. In an embodiment, the middlecolumn of pixels of a reference-line referenced image correspond to theposition of the linear feature taken from the existing map database. Ifthe position of the linear feature is correct, the linear feature willbe visualized in the middle column and thus be seen as a straight line.However, if the position in the database is incorrect, the linearfeature will not be visualized in the middle column of pixels in thereference-line referenced image. Furthermore, if a linear feature is nota straight line in the reference-line referenced image, this is anindication that the reference-line is not digitized correctly in thedigital map database. A human will recognize both situations as a defectin the digital map database and could mark the defect in thereference-line reference image. Subsequently, the geo-position of thedefect corresponding to the position of the mark in the reference-linereferenced image is determined and stored in a database. Thegeo-positions of the defects can be used to retrieve correspondingsource images from a mobile mapping vehicle or a correspondingorthorectified image, to digitize again the linear feature in imageswith a high 2D-resolution, such as an orthorectified image. In this way,data in existing map databases can be verified efficiently and correctedif necessary. The verification method can be improved by taking as areference line the direction of the road, e.g. centerline, andsuperposing the positions of linear features, such as road width, roadmarkings, in the reference-line referenced image. An advantage of thereference-line referenced image is that a human has to scroll only upand down in the image to move along the road. Furthermore, theresolution along the road (linear feature) could be varied withoutlosing position accuracy in a direction perpendicular to the directionof the road. This is elucidate in FIG. 13 and corresponding description.

In block 414 the position in the geographic coordinate reference systemof the selected linear feature in block 412 is calculated from thexy-position of the linear feature in the reference-line referencedimage. The projected coordinate reference system associated with eachpixel defines the coordinate conversion from reference-line referencedimage space to geographic coordinate reference system space. Finally,the linear feature and corresponding position in the geographiccoordinate reference system is stored in a database for use in a mapdatabase.

In the following paragraphs the mathematics of reference-linereferencing will be given.

Let n be a natural number, r a natural number or ∞, I be a non-emptyinterval of real numbers and t in I. A vector valued functionγ:I→

of class Cr (i.e. γ is r times continuously differentiable) is called aparametric curve of class Cr or a Cr parameterization of the curve γ. tis called the parameter of the curve γ. γ(I) is called the image of thecurve.

For a 2-dimensional space, e.g. an image, such curve will therefore havefollowing formγ(t)=[x(t),y(t)]

For the C1 class curve at each point we can define it's tangent andnormal vectors which can be computed from following equation:

The unit tangent vector is the first Frenet vector e₁(t) and is definedas

${e_{1}(t)} = \frac{\gamma^{\prime}(t)}{{{\gamma^{\prime}(t)}}}$

A normal vector, sometimes called the curvature vector, indicates thedeviance of the curve from being a straight line.

The normal vector is defined ase ₂ (t)=γ″(t)−

γ″(t),e ₁(t)

e ₁(t)

Its normalized form, the unit normal vector, is the second Frenet vectore2(t) and is defined as

${e_{2}(t)} = \frac{\overset{—}{e_{2}}(t)}{{{\overset{—}{e_{2}}(t)}}}$

Each point in the neighbourhood of the line can be represented asreference-line referenced in the following form:P(t,o)=γ(t)+o*e2(t)

This principle can be applied to any pixel on an image that isgeoreferenced and is in neighbourhood of the line.

The reference-line referenced images transformation algorithm isillustrated in FIG. 6 and comprises the following actions:

-   -   1. Let's define d(n) 610 as a sequence of distance along        reference geometry points 612, where n is ordering number in        sequence, so that d(n)<d(n+1) and 0<=d(n)<=G, where G is        reference geometry width.        -   Reference geometry is sampled with distances along resulting            from vector d(n). Value R(n)=d(n)−d(n−1) can be treated as            local resolution in first or horizontal axis of linear            reference image.    -   2. For each element from vector d(n) perform following:        -   2.1. Query point and geometry direction vector p 608            perpendicular to reference geometry 602 at distance D=d(n);        -   2.2. Create source image sampling line 604 using direction            vector p 608 and given sample line width 606. Sample this            line with given step. As a result we get sequence of points            610. Let we assume that result points Pt(n, m) are ordered            from left to right in direction according to geometry            direction, where n is the index of current element of vector            d(n) and m is ordering number of point at reference-line.        -   2.3. For each of the points Pt(n, m) created in pt. 2.2:            -   2.3.1. Using well known spatial reference system                conversions to find the pixel (or group of pixels)                corresponding to point Pt(n, m) on source orthorectified                image.            -   2.3.2. Perform any image processing operations on found                pixel (or group of pixels) to get result pixel value.            -   2.3.3. Set pixel value resulting from 2.3.2. into result                reference-line referenced image at pixel location (n,                m).

In other words, the reference-line, which is the reference geometry, forexample the road centreline or track line of a vehicle, is sampled toobtain reference-line points. Then for each reference-line point, thegeometry direction vector p 608 perpendicular to the reference-line isdetermined. The reference-line points and the geometry direction vectorare used to determine the geo-position of a line section on theorthorectified image. The line section is on a line having a directionwhich is perpendicular to the direction of the reference-line in a pointwhere the line through the line section crosses the reference-line. Theline section is used to generate a row of pixels. The row of pixelscorresponds to geo-positions on a line. The geo-positions areequidistant and form a straight line of a reference-line referencedimage. The geo-positions are used to derive the corresponding positionof pixels in the geo-referenced orthorectified image. The value of apixel of the row of pixels is determined from the values of thecorresponding pixels in the geo-reference orthorectified image. Theresolution of the row of pixels is in an embodiment 0.5 cm. However, therequired resolution depends on a combination of the dimensions of thelinear road information to be detected and the required position anddimensions accuracy in the map database. For example, the resolution ofthe road width will be defined by the required position accuracy,whereas the width of a road centreline could be defined by the width ofthe line.

The rows of pixels corresponding to subsequent reference-line points canbe combined to form a reference-line referenced image. In areference-line referenced image, the position of the reference-line isprojected on one of the columns of pixels. Each other column of pixelsin the reference-line referenced image corresponds to a line at apredefined perpendicular distance from the rack line.

In FIG. 6 is shown the direction of the reference-line 602 in a specificreference-line point 612. Perpendicular to the direction 602 is a linesection 604. The line section has a predetermined width 606 anddirection vector 608. Along the line section, a number of line sectionpoints 610 are selected. Each line section point 610 has a correspondingpixel in a row of pixels. A row of pixels can be stored as an array ofpixels but also as a row of pixels in a reference-line referenced image.Each line section point has a geo-position. Preferably, one line sectionpoint has a geo-position which is at the position where the line section604 crosses the reference-line. However, the invention could also workproperly when the area of the line section does not cover thereference-line. The only constraint is to produce linear roadinformation is that the geo-positions of the road information isparallel to the geo-position of the reference-line.

The rows of pixels corresponding to the reference-line points are storedin a reference-line referenced data set. Each row of pixels can storeindividual sets of data comprising image data representative of the rowof pixels and position data, representative of the position andorientation of the reference-line with respect to a reference pixel ofthe row of pixels. By means of the position data, the geo-position ofevery pixel of the row of pixels can be determined accurately. In thisway, each the geo-referenced position of a pixel can be defined usingthe following values: the distance along the reference-line and thedistance from the reference geometry in perpendicular direction to thedirection of the reference-line.

FIG. 7 illustrates the generation of a reference-line referenced imagefrom an orthorectified image. FIG. 7 shows left an orthorectified imageof a road 702. A reference-line 710 is superposed over the road 702.Lines sections 706 are shown. The line sections 706 have a directionwhich is perpendicular to the direction of the reference-line 710 wherethe line sections 706 crosses the reference-line 710. FIG. 7 shows atthe right side a reference-line referenced image 704. Line 712superposed on the reference-line referenced image 704 corresponds to thereference-line 710. The arrows 707 indicates the mapping from a linesection 706 in the orthorectified source image to a row of pixel 706 ofan reference-line referenced image 704. It can be seen that the lineshaving a constant perpendicular distance in an orthorectified image 702are transformed to parallel “vertical” lines in the reference-linereferenced image 704.

FIG. 7 shows an example wherein the distance along the reference-linebetween subsequent reference-line points 709 is equidistant. In anembodiment the distance between two reference-line points along thereference-line is 1 meter. This has the advantage that the linearreference image generated by combining the rows of pixels, is easilyinterpretable. It has been found that a distance between tworeference-line points along the reference-line in the range of 0.08 m-2m provides easy to interpret reference-line referenced images. Thelowest distance corresponds to the resolution of the pixels of a linesection, which depends on the image resolution of the source images ororthorectified images. Every predefined displacement along a columncorresponds to the same displacement of the line section along thereference-line. If a human has to mark the linear features in thereference-line referenced image which cannot be visualized on onescreen, the human has to scroll vertically through the reference-linereferenced image. A vertical scroll of a number of lines of pixels willcorrespond to a movement of a corresponding distance along thereference-line.

The distance between two reference-line points along the reference-linein a reference-line referenced image could be increased further.However, in that case some pre-filtering is needed to make the imageinterpretable. If no pre-filtering is performed, linear features, suchas dashed painted lane dividers could disappear or shown as solid lines.In an embodiment of a pre-filter, if an engineer would like to have e.g.8 m spacing between two reference-line points, reference-line referenceddata with a spacing of 1 m is generated. Each line in the reference-linereferenced image is obtained by filtering 8 subsequent line sectionsalong the reference-line. Each even line of the reference-linereferenced image is obtained by storing the minimal luminance value ofthe pixels of the 8 subsequent line section at equivalent perpendiculardistance from the reference-line point and each uneven line of thereference-line referenced image is obtained by storing the maximalluminance value of the pixels of the 8 subsequent line section atequivalent perpendicular distance from the reference-line point. In thatcase a human would be able to see dashed painted dividers on the imageas a vertical bar with alternating black (minimal luminance) and white(maximal luminance) pixels. Larger distances between two reference-linepoints can be very suitable for visually verifying the geo-position oflong linear features in an existing digital map database, such as lanedividers on a high way. By the given embodiment, instead of the screendisplaying 1 km along the road it will display 8 km. To verify a roadwith a length of 80 km only 10 screens have to be verified instead of80. Furthermore, the human interactions are reduced with a factor of 8,which improves the throughput time for verifying the linear roadfeatures of said 80 km road.

If the reference-line corresponds to the track-line of a moving vehiclealong the road, the distance between two reference-line points dependson the maximum deviation of the heading direction of the vehicle withrespect to the direction of the road and thus the linear features alongthe road. A deviation of the direction of the vehicle with respect tothe direction of the true linear object will result in a slanted linearfeature in a reference-line referenced image. FIG. 10 shows an example.For these applications it is assumed that the track-line and the linearroad feature are not more then 5 degrees off. This allows us to samplealong the reference-line, which defines the vertical resolution of areferenced-line referenced image, twelve times less frequent inorthorectified space than the target resolution of the line sectionperpendicular to the reference-line, which is the horizontal resolution.Optimal sampling along the reference-line should allow us to achieve thesame expected digitization error along as well as across thereference-line. The error is proportional to res_hor/tan(alpha_diff)wherein alpha_diff is the local difference of direction of thereference-line and the direction of the linear feature, and res_hor isthe horizontal resolution of the reference-line referenced image.Optimal sampling can be achieved via an assumption (maximal expectedangle between direction of reference-line and direction of linearfeature) or via image recognition (which determines the angleautomatically), wherein the angle of a recognized linear feature in thereference-line referenced image with respect to a column of pixelsdefines the sampling to generate the subsequent reference-linereferenced image along the reference-line. Image recognition detecting achange of lane could be used to improve further the optimal sampling. Ifthe reference-line corresponds to the track line of a vehicle driving ona highway, the vehicle could follow the road direction or could changeof lane. A change of lane will result in a curve in representation ofthe linear feature in the reference-line referenced image, which has tobe digitized correctly in the image. To do this the distance between twopoints along the reference-line has to be small enough. If thetrack-line follows correctly the direction of the road, the linearfeature will still be a straight feature in the reference-line referenceimage and the distance between two sample points needs not to bechanged.

In another embodiment, the distance depends on the winding of thereference-line. If a reference-line is more or less straight, fewerpoints are needed to describe accurately the curvature then when thereference-line is curvy. This embodiment is useful to reduce the amountof data in a reference-line referenced data set. This could haveadvantages in the event the linear features are detected in thereference-line referenced data automatically by commonly known linedetection algorithms.

FIG. 8 shows an example of the reference-line referenced imagestransformation algorithm wherein the reference-line 802 is not acrossthe road but parallel to the outside bend of the road. Arrow 806indicates the direction of the reference-line 802 in the orthorectifiedimage and arrow 808 indicates the direction of the reference-line 804 inthe reference-line referenced image. Similarly, FIG. 9 shows an exampleof the reference-line referenced images transformation algorithm whereinthe reference-line 902 is not across the road but parallel to the insidebend of the road. Arrow 904 indicates the direction of thereference-line 902 in the orthorectified image and arrow 908 indicatesthe direction of the reference-line 906 in the reference-line referencedimage. FIG. 10 shows an example of the reference-line referenced imagestransformation algorithm wherein the orientation of the reference-line1006 is angled with the orientation of the road. Arrow 1004 indicatesthe direction of the reference-line 1004 in the orthorectified image andarrow 1008 indicates the direction of the reference-line 1006 in thereference-line referenced image. It can be seen in the linear referenceimage at the right hand side, that the reference-line is vertical andthe linear features, centerline 1010 and road side 1012 are slantvertical lines. To mark accurately the track of the centerline and roadsides in orthorectified images shown in FIGS. 8-10, multiple marks haveto be placed on the linear features. However to mark the track of thecenterline and road sides in the reference-line referenced images shownin FIGS. 8-10, only the beginning and ending point of the centerline andtwo road sides in each image have to be marked. By knowing thegeo-referenced position (in real world coordinates) of each pixel of thereference-line referenced image, the geo-position of the reference-lineof the linear features can be determined accurately. Therefore, the useof reference-line referenced images with associated map projectionenables us to speed-up the manual marking of linear road features togenerate linear features for use in a map database.

In view of the above a reference-line referenced image has the followingproperties:

-   -   Total number of points used to indicate continuous linear        elements of road (edges, lane dividers, etc.) is much fewer on        reference-line referenced images, than the number of points that        are necessary to indicate the same elements on source        orthorectified image, with assumed equal relative and absolute        accuracy of resulting curve. As each point (pixel) of the        reference-line referenced image is unambiguously convertible to        the projected coordinate referenced system of the orthorectified        source image, it is possible to convert curves from        reference-line referenced image to real world curves in the        geographic coordinate reference system.    -   The total number of points needed to indicate a continuous        linear object on a reference-line referenced image depends on        the change of angle between the reference geometry and the        object on the orthorectified source image. The more constant the        angle, the fewer points should be marked to achieve the required        accuracy of the object in the reference-line referenced image.        The change of angle defines the straightness of the linear        feature in the reference-line referenced image.

FIG. 11 illustrates the advantage of the invention. At the left handside, an orthorectified image of a curved road is shown. To mark thecenterline of a road accurately, 14 marks have to be placed. At theright hand side, a reference-line referenced image is showncorresponding to the curved road in the left hand side image. To obtainthe reference-line referenced image, the reference reference-line wasparallel to the centerline of the road. In the reference-line referencedimage only two marks are needed to mark accurately the centerline in theimage. Thus, instead of 14 marks, now only two marks have to be placed,the beginning point 1106 and the ending point 1104 of the centerline inthe image.

FIG. 12 illustrates the transformation from reference-line referencedimage space back to orthorectified image space of the centerline markedin the reference-line referenced image shown in FIG. 11. Line 1200corresponds to the position of the centerline in the reference-linereferenced image. Each pixel of the reference-line referenced image canbe unambiguously converted to a position in the orthorectified image.Therefore, the line between beginning point 1206 and ending point 1204across the reference-line referenced image can be unambiguouslyconverted to a curve on the orthorectified image. As each pixel in theorthorectified image has a geo-referenced position, the geo-referencedposition of the curve can be accurately determined.

FIG. 13 shows a further advantage of a reference-line referenced image.In a reference-line referenced image, a curved reference-line on aorthorectified image will be a straight line. Furthermore, from FIG. 10it is learned that even if the reference-line makes a constant anglewith the linear feature in the orthorectified image, the linear featurewill appear as a slanted straight line in the reference-line referencedimage. This provides the possibility to change the vertical resolutionof the reference-line referenced image without losing positioninformation of the linear features. By decreasing the number ofreference-line points along a reference-line in the reference-linereferenced image, the vertical size of the image reduces. Consequently,a section along the reference-line will be smaller while the resolutionperpendicular to the reference-line will be maintained in the reducedlinear reference image. The left hand reference-line referenced image1302 shows a road segment with a first resolution along thereference-line and the right hand reference-line referenced image 1304shows the same road segment but with a resolution along thereference-line which is half of the first resolution. A change ofresolution can be seen as a change in vertical scale of thereference-line referenced image. The allowable scale factor depends onhow much deviation of angle between road objects and reference geometrychanges, can be expected. The error introduced by such scaling willproportionally increase by the tangent angle between two linearfeatures.

The embodiments and examples of the invention described above usegeo-referenced orthorectified images as image data source andgeo-referenced reference-line data. For each reference-line point, theposition of a line section in the orthorectified image perpendicular tothe direction of the reference-line in the reference-line point isdetermined. The position is used to derive a row of pixels for said linesection from the corresponding pixels in the orthorectified image.However, it should be noted that the source images of the moving vehiclecould also be used directly to derive values for the row of pixels. Assaid above, for each pixel the corresponding position in a geographiccoordinate reference system is known. Unpublished patent applicationPCT/NL2006/050252 teaches how the x,y coordinates of pixels of thesource image can be converted to the x,y position of a pixel of aorthorectified tile. Therefore, it is possible to convert directly thex,y coordinates of pixels in the source images obtained by a cameramounted on a moving vehicle corresponding to pixels of a row of pixelsof a line section perpendicular to the direction of the reference-lineat a reference-line point. Thus the invention can use an image datasource wherein for each source image the characteristics of the camera,such as angle of view and focus length, and the position and orientationin a coordinate system is known. Therefore, the raw data captured by amobile mapping vehicle is very suitable to produce a reference-linereferenced image. The raw data comprises image sequences and positionand orientation data of the mobile mapping vehicle in a geographiccoordinate reference system. This information is enough to determine thex,y coordinates of pixels in a source image that correspond to a pixelin the reference-line referenced image. Furthermore, as the mobilemapping vehicle drives on the road, the recorded position information issuitable to be used as reference-line data.

The invention is very useful to producing linear road information.However, the invention can also be used to verify the linear roadinformation in existing digital map databases. For each pixel in areference-line referenced image its corresponding geo-coordinate isknown. This makes is possible to overlay the linear road information inan existing digital map on the reference-line referenced image. Byscrolling the linear reference image with overlaid linear roadinformation along the reference-line, a human can very easily verifyvisually whether the linear road information in the existing map iscorrect and accurate enough. If not, the human just has to move theposition of the overlaid linear road information on the reference-linereferenced image and the updated positions of the road information canbe stored for use in the digital map.

The present invention allows a human to analyze 20 times moreinformation at one time then originally when looking at orthorectifiedimages. The transformation to reference-line referenced images saves amultitude of screen scrolling needed to view next portions of a roadsegment by 7 times. Furthermore, the number of necessary points todigitize linear features is reduced by 20 times.

The method of the invention can be implemented on a computer arrangementas shown in FIG. 3.

The foregoing detailed description of the invention has been presentedfor purposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise form disclosed, andobviously many modifications and variations are possible in light of theabove teaching. The present invention can also be used without anymobile mapping system data. In an embodiment, the input images areaerial images with sufficient detail to detect linear road information,and the reference-line data corresponds to the centerline of a road froman existing database. From, those input data source a reference-linereferenced image can be generated for use in the applications describedabove, e.g. producing linear road information or verifying the positionof existing linear road information in an existing map database and tocorrect the information if necessary.

The described embodiments were chosen in order to best explain theprinciples of the invention and its practical application to therebyenable others skilled in the art to best utilize the invention invarious embodiments and with various modifications as are suited to theparticular use contemplated. It is intended that the scope of theinvention be defined by the claims appended hereto.

1. A method of capturing linear features along a reference-line across asurface for use in a map database, comprising: generating, fromreference-line data representative of coordinates of said reference-linein a geographic coordinate reference system and source images of asurface adjacent to said reference-line and associated position andorientation data in said geographic coordinate reference system, areference-line referenced data set, wherein the reference-linereferenced data set comprises a plurality of sets of image data andassociated data defining a reference-line across a surface in thegeographic coordinate reference system, the sets of image data includingpixels wherein a set of image data corresponds to an orthorectified viewrepresentation of a line section of the surface in the geographiccoordinate reference system, each set of image data comprises areference pixel associated with a position on the reference-line,wherein each pixel represents a surface including a position at adistance from the position of the reference pixel along the linesection, and wherein the line section perpendicularly crosses thereference-line at the position associated with the reference pixel; andpost processing the generated reference-line referenced data set toproduce linear features along the referenceline and associated locationsin the geographic coordinate reference system for use in a map database.2. The method according to claim 1, wherein the source images include atleast one of: images captured by a terrestrial camera mounted on amoving vehicle, aerial images, satellite images, and orthorectifiedimages.
 3. Method according to claim 1, wherein the reference-lineincludes at least one of: track line of a vehicle, road centerline fromexisting database, and other existing linear road geometry taken from anexisting database.
 4. The method of post processing a reference-linereferenced data set to produce linear features along a reference lineacross a surface and associated locations in a geographic coordinatereference system for use in a map database, the method comprising;retrieving the reference-line referenced data set, wherein thereference-line referenced data set comprises a plurality of sets ofimage data and associated data defining a reference-line across asurface in the geographic; coordinate reference system, the sets ofimage data including pixels wherein a set of image data corresponds toan orthorectified view representation of a line section of the surfacein the geographic coordinate reference system, each set of image datacomprises a reference pixel being associated with a position on thereference-line, wherein each pixel represents a surface including aposition at a distance from the position of the reference pixel alongthe line section, and wherein the line section perpendicularly crossesthe reference-line at the position associated with the reference pixel;transforming the reference-line referenced data set into areference-line referenced image, wherein each column of pixels of thetransformed image corresponds to a surface parallel to the direction ofsaid reference-line; selecting a linear feature in the reference-linereferenced image: determining coordinates in the geographic coordinatereference system of the linear feature from the position of the linearfeature in the reference-line referenced image and associated data; andstoring the coordinates of the linear feature in a database.
 5. Themethod according to claim 4, wherein the linear feature includes atleast one of: road centerline, road width, road curb, road painting,lane divider, number of lanes, traffic island, junctions and any othervisual distinguishing feature of the surface along the reference-line.6. The method according to claim 4, wherein the selecting comprises:outputting the reference-line referenced image on a screen; and manuallypositioning a pointing device on the linear feature on the screen toobtain marked positions to define a position of the linear feature inthe reference-line referenced image; and determining the coordinates inthe geographic coordinate reference system of the linear feature fromthe position of the marked positions in the reference-line referencedimage.
 7. The method according to claim 4, wherein the selectingcomprises: performing a line detection algorithm on the reference-linereferenced image to select the linear feature; determining the pixe1position of linear feature in the reference-line referenced image; anddetermining the coordinates in the geographic coordinate referencesystem of the linear feature from the pixel positions of the linearfeature in the reference-line referenced image.
 8. A computerimplemented system for performing post processing of a reference-linereferenced data set to produce linear features along a reference-lineand associated locations in a coordinate system for use in a mapdatabase, the system comprising: an input device; a processor readablestorage medium: a processor in communication with said input device andsaid processor readable storage medium; and an output device to enablethe connection with a display unit, said processor readable storagemedium storing code to program said processor to perform actionscomprising: retrieving the reference-line referenced data set, whereinthe reference-line referenced data set comprises, a plurality of sets ofimage data and associated data defining a reference-line across asurface in a geographic coordinate reference system, the sets of imagedata including pixels wherein a set of image data corresponds to anorthorectified view representation of a line section of the surface inthe geographic coordinate reference system, each set of image datacomprises a reference pixel being associated with a position on thereference-line, wherein each pixel represents a surface including aposition at a distance from the position of the reference pixel-alongthe line section, and wherein the line section perpendicularly crossesthe reference-line at the position associated with the reference pixel;transforming the reference-line referenced data set into areference-line referenced image, wherein each column of pixels of thereference-line referenced image corresponds to a surface parallel to thereference-line; selecting a linear feature in the reference-linereferenced image; determining coordinates in the geographic coordinatereference system of the linear feature from the pixel position of thelinear feature in the reference-line referenced image and associateddata; and storing the coordinates of the linear feature in a database.9. A computer implemented system for verifying linear features along areference-line and associated locations in a coordinate system, thesystem comprising: an input device: a processor readable storage medium;a processor in communication with said input device and said processorreadable storage medium; and an output device to enable the connectionwith a display unit, said processor readable storage medium storing codeto program said processor to perform actions comprising: retrieving froma map database reference-line data representative of coordinates of areference-line across a surface in a geographic coordinate referencesystem; retrieving an orthorectified image of said surface andassociated position and orientation data in said geographic, coordinatereference system generating from the reference-line data andorthorectified images a reference-line referenced image, wherein eachrow of pixels of the referenced-line referenced image corresponds to asection of said surface perpendicular to the direction of thereference-line and each column of pixels of the reference-linereferenced image corresponds to a surface parallel to thereference-line; verifying the position of linear features in thereference-line referenced image; marking positions showing defects withrespect to at least one of position of linear feature, straightness oflinear feature, and parallelism of linear features; determiningcoordinates in the geographic, coordinate reference system of the markedpositions in the reference-fine referenced image showing defects; andstoring the coordinates of the marked defects in a database for furtherprocessing.
 10. A non-transitory processor readable medium carrying acomputer program product, which when executed on a computer arrangement,allow said computer arrangement to perform the methods according toclaim 1.